As the demand for computing resources has increased exponentially over the past decade, new ways have been sought to allow computing systems to process large amounts of data and user requests in an efficient manner.
A common way to handle a large number of simultaneous user requests on a single computing system is to divide the requests either by software or by hardware, into separate “tasks” or “transactions”. In a software environment, this has become known as multi-tasking, a method whereby simultaneous user requests are placed in a queue, and executed in a sequential manner by a process. A transaction request will be understood to be a request generated by a software application, for some task to be performed by a computing system. A process will be understood to be a component (usually software) within an operating system which accepts a transaction request (hereinafter referred to as a “transaction”), queues the transaction, and then processes the transaction. Processing the transaction will be understood to mean carrying out a task or tasks requested by the transaction. The task and/or tasks may include any suitable instructions, such as searching a database, carrying out a mathematical calculation, writing to a file, accessing a hardware device, or any other activity which may be performed by a computing system. In a hardware environment, the ability to handle multiple simultaneous user requests has commonly been enhanced by using more than one central processing unit (processor).
When constructing operating systems, or other software designed to run in a multi-tasking environment, programmers are faced with the issue of deciding how transactions are allocated amongst several processes. The simplest way to decide which process a transaction will be sent to is to allocate the process in a round robin fashion. That is, a transaction currently waiting in the queue would be allocated to the next available process.
Alternatively, other operating systems allocate transactions randomly to any process, irrespective of its availability or other operating characteristics.
Such allocation systems have been found by the present applicants to be sub-optimal, because they do not take into account the time overhead involved in flushing various memory caches used for the previous process, and restocking various memory caches before the new transaction can be processed.
When a transaction is scheduled to be executed by a process that has been idle for some time rather than by a process that has recently completed executing a transaction, there is a high likelihood that the code and data used by the previous transaction has been flushed from the process's associated cache (for example, the cache in a processor used by the process). Therefore, the probability of cache misses is increased and performance suffers accordingly.
In addition, when a transaction is scheduled to a process that has been idle for some time rather than to a process that has recently become available, the likelihood that the process's associated virtual memory has been paged out to disk is also increased. Therefore, the probability of page faults occurring is increased and performance suffers accordingly. A page fault will be understood to mean an error which occurs when a program requests data that is not currently loaded into memory. This requires an operating system to fetch the requested data from a storage device and load it into memory.
Similarly, the same performance degradation may occur with any type of associated cache memory whether it be processor cache, main system memory, or virtual memory on a data storage device, such as a hard disk.
The problem of having a high number of process cache misses or other secondary cache misses due to the random scheduling of incoming transactions to any available process is not addressed by the prior art. It has generally been assumed by programmers that the operating system which resides on the computing system will schedule the transactions to processes in the most efficient manner possible. This assumption may be incorrect for certain applications.
While it is true that some operating systems have better scheduling strategies than others, in general the operating system is not aware of the specific architecture of the applications it is running, and therefore it cannot fully optimise the transaction scheduling for these applications.
In the past, the problem of having increased page faults due to the scheduling of incoming transactions to processes that may have already had their virtual memory swapped out to disk is generally addressed by adding more RAM (Random Access Memory) to a system. While this works well in general, it increases the hardware requirements for the system. Thus, adding RAM is also a sub-optimal solution as it does not address the fundamental nature of the problem. Therefore, there is a need for a new system and method which allows for more efficient use of computing resources.